Multi-user application system environment engine

ABSTRACT

A multi-user application system environment engine has an application system that, in turn, includes a simulation engine and a virtualized software environment. The simulation engine runs on top of the virtualized software environment and includes a declaration processor, a scene tree object manager, a persistence processor in communication with the scene tree object manager, a visual editor, an editor broadcaster, an editor listener, and a rendering processor, coupled to the virtualized software environment, to requisition hardware resources to cause physical manifestation of an instantiated scene tree of objects.

RELATED APPLICATION

The present application has subject matter related to an internationalapplication under the Patent Cooperation Treaty having internationalpublication number WO2016/063022. That related application, for aninvention by the same inventor as for the present application, is herebyincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to application systems, and moreparticularly to application systems in a multi-user environment.

SUMMARY OF THE EMBODIMENTS

Embodiments of the present invention provide an environment that differsfrom prior art computer-focused environments requiring tools, modulesand programming languages as conditions for causing computer executionof programs. Embodiments of the present invention provide an environmentin which computer processes are made subservient to a human-oriented setof requirements and goals. The requirements and goals are specified at ahigh level by the user in a manner independent of the hardware,operating system and software libraries, and it is then up to thecomputer software to work out how to meet these requirements and goals.In embodiments of the present invention, the requirements and goals arespecified by the user in a new layer, which we call an “applicationsystem,” wherein the interaction is accomplished using s simulationengine. The simulation engine can exploit many pre-compiled highlyefficient subsystems with lazy evaluation, domain specific heuristicsand machine learning to achieve the goals extremely efficiently.Importantly, unlike normal software, which requires just in time orstatic compilation before it is run, the application system provides, inits user interface, what we call “declarations,” which can be updatedlive while they are being simulated.

In accordance with one embodiment of the invention, a multi-userapplication system environment engine has an application system that, inturn, includes a simulation engine and a virtualized softwareenvironment. The simulation engine runs on top of the virtualizedsoftware environment and includes a declaration processor, a scene treeobject manager, a persistence processor in communication with the scenetree object manager, a visual editor, an editor broadcaster, an editorlistener, and a rendering processor, coupled to the virtualized softwareenvironment, to requisition hardware resources to cause physicalmanifestation of an instantiated scene tree of objects.

More particularly, in one embodiment, there is provided a nontransitorystorage medium encoded with instructions, for use with a plurality ofcomputer systems, each one of which is in communication in real timeover a network with the other computer systems, wherein theinstructions, when executed on each of the computer systems, establish,on each computer system, an application system. The application systemincludes a simulation engine and a virtualized software environment, andthe simulation engine runs on top of the virtualized softwareenvironment and enables creation and editing of a project by a localuser and network users. In this embodiment, the simulation engineincludes a declaration processor that linearly processes a firstdeclaration that constitutes a text embodiment of the project, recitingany set of items selected from the group consisting of object,relationship, property, behavior, and combinations thereof, into adeclared scene tree of objects.

The application system of this embodiment additionally includes a scenetree object manager that (a) manages objects, their relationships,properties, and behaviors in a declared scene tree of objects, (b)causes transformation of the declared scene tree of objects so as to bemirrored in a project subnode of an instantiated scene tree of objects,the project subnode constituting an instantiated embodiment of theproject, and (c) causes updating of the instantiated scene tree when itis changed.

The application system of this embodiment additionally includes apersistence processor in communication with the scene tree objectmanager, that causes mirroring, of the instantiated scene tree ofobjects, in a second declaration. The application system furtherincludes a visual editor, the visual editor being a subnode of theinstantiated scene tree, that makes user-directed changes to the projectsubnode of the instantiated scene tree, wherein the project subnode is asubnode of the visual editor.

Furthermore, the application system of this embodiment includes aneditor broadcaster and an editor listener. The editor broadcaster is asubnode of the instantiated scene tree, coupled to the network, thatobtains changes from the visual editor and causes the persistenceprocessor to transform a correspondingly changed part of theinstantiated scene tree into a change declaration and broadcasts thechange declaration embedded in a change message over the network to theother computer systems. The editor listener is a subnode of theinstantiated scene tree, coupled to the network, that receives andprocesses any change message, from any other of the computer systems, bycausing the declaration processor to transform a declaration embedded inreceived change message into a corresponding change in the instantiatedscene tree.

Finally, the application system of this embodiment includes a renderingprocessor. The rendering processor is coupled to the virtualizedsoftware environment, to requisition hardware resources to causephysical manifestation of the instantiated scene tree of objects in aform selected from the group consisting of audio, visual, networkcommunication, storage in a transitory or a nontransitory basis,computational tasks, and combinations thereof, the physicalmanifestation being optimized by evaluation of hardware resources thatare both available and required to be allocated in order to achieve thephysical manifestation.

In a related embodiment, the editor has a mode wherein it operatessolely as a viewer and does not communicate with the editor broadcasteror the persistence processor. Alternatively or in addition, the visualeditor maintains a globally unique identifier for each object in theinstantiated scene tree, so that each change, made over the network, canuse the identifier to reference the corresponding object and that suchchange can be correctly applied to the instantiated scene tree. Alsoalternatively or in addition, the visual editor associates, with eachchange made by a user over the network, an identification of the usermaking the change and a time stamp. Also alternatively or in addition,the visual editor characterizes each change, made over the network, tothe instantiated scene tree, as a member of the group consisting ofinsertion, deletion, update, lock, unlock, and combinations thereof.Also alternatively or in addition, each of the computer systems iscoupled over the network to a server system, and the editor broadcasteralso broadcasts the change declaration embedded in the change messageover the network to the server system for storage thereof in adeclaration repository. Also alternatively or in addition, thesimulation engine further includes a frame clock defining consecutiveframes of the engine with respect to which operations are synchronized.

In another related embodiment, the simulation engine further includes aframe processor, coupled to the virtualized software environment and tothe simulation engine, to require the simulation engine to performoperations, specified with respect to a sequence of frames using theframe clock, to assure that the processes are performed sequentially.Optionally, (a) the simulation engine further comprises a frame queue,filled by the simulation engine, in which are stored, in frame order,software contracts pertinent to processes to be carried out by thesimulation engine, and (b) the frame processor is coupled to the framequeue and, if a specific software contract in the frame queue reaches acurrent frame for performance, the frame processor enforces the terms ofthe specific software contract as to best efforts and guaranteed terms.In a further related embodiment, the frame processor is configured toprocess, concurrently with the current frame, a set of frames sequencedafter the current frame, with respect to demand expected to be placed onsystem resources by the specific software contract, so as to enhanceefficiency of execution of the contract when each frame in the set offrames shall have become the current frame. As a further option, in theprocessing of an event, via the frame processor, from the virtualizedsoftware environment, the simulation engine executes logic creating atemporal assignment, which is added to the frame queue based on a framenumber computed, based on a current frame number and a target time, inframe order by object and property, so that if the computed frame numberis less than a frame number of any second item in the frame queue, thenthe second item will be flushed from the queue. Also optionally, at thetime of creating the temporal assignment the frame processor stores aninitial set of values and the initial time along with a target set ofvalues and a target time, and an interpolation function, when a frameneeds to be computed, the frame processor computes a set of currentvalues using the current time, the initial time and the target time todrive the interpolation function between the initial values and targetvalues.

In a further related embodiment, the frame processor operates on a bestefforts basis to cause updating of the interactive scene tree and visualscene tree so as to correspond with the calculated set of values untilthe final frame has been reached, at which time the frame processorcauses the temporal assignment to be contractually updated throughoutthe system. As a further option, in the processing of an event, via theframe processor, from the virtualized environment, the simulation engineexecutes logic creating a temporal procedure, which is added to theframe queue at a frame number computed, based on a current frame numberand a target time, in frame order by object and property, so that if thecomputed frame number is less than a frame number of any second item inthe frame queue, then the second item will be flushed from the queue.

In another related embodiment, the first declaration conforms to adeclaration grammar wherein:

a loop is not permitted;

a branching condition is not permitted;

a procedure with a set of parameters is permitted; and

an object in a declared scene tree is permitted to have a set of states,wherein:

-   -   each state must be separately selected from a group consisting        of (a) a state that is a set of nested states and (b) a state        that is nonoverlapping with any other state; and    -   each state can encompass a set of properties, a set of events,        and a set of sub-objects; and    -   a given state can only be active or inactive; and    -   active states in a declaration are applied, by the scene tree        manager, in the order in which they are declared.

Optionally in the immediately preceding related embodiment, thepermitted procedure with the set of parameters includes a procedure callto a procedural function in a scripting language.

In another related embodiment, the scene tree is configured to cause thesecond declaration is to replace the first declaration. Alternatively orin addition, the simulation engine includes a library of classes thatimplement an abstraction of a set of operating systems. Alternatively orin addition, the editor listener of any given client includes anoff-line synchronizer coupled to a repository server system, therepository server system having a declaration repository that stores alog of each change made in any declaration by any user on the network toany scene tree accessible over the network, so that, when the givenclient comes online after having been offline, the off-line synchronizercommunicates with the declaration repository to synchronize changes thatwere not synchronized when the given client had been offline.

In another embodiment, the invention provides a nontransitory storagemedium encoded with replicating instructions, for use on a computerconfigured as a replicating computer in the computer environmentdescribed with respect to the previous embodiments, the replicatinginstructions, when executed on the replicating computer, establish, onthe replicating computer, a replicating application system, wherein thereplicating application system includes a replicating simulation engineand a replicating virtualized software environment, the replicatingsimulation engine running on top of the replicating virtualized softwareenvironment. In this embodiment, the replicating simulation enginecomprises: a replicator declaration processor that linearly processes afirst declaration that constitutes a text embodiment of the project,reciting any set of items selected from the group consisting of object,relationship, property, behavior, and combinations thereof, into adeclared scene tree of objects; a replicator scene tree object managerthat (a) manages objects, their relationships, properties, and behaviorsin a declared scene tree of objects, (b) causes transformation of thedeclared scene tree of objects so as to be mirrored in a project subnodeof an instantiated scene tree of objects, the project subnodeconstituting an instantiated embodiment of the project, and (c) causesupdating of the instantiated scene tree when it is changed; a replicatoreditor, that makes user-directed changes to the project subnode of theinstantiated scene tree, wherein the project subnode is a subnode of thevisual editor; and a replicator listener, coupled to the network, thatreceives and processes any change message, from any other of thecomputer systems, by causing the declaration processor to transform adeclaration embedded in received change message into a correspondingchange in the instantiated scene tree.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood byreference to the following detailed description, taken with reference tothe accompanying drawings, in which:

FIG. 1 is a flow diagram showing logical processes carried out by asimulation engine in accordance with an embodiment of the presentinvention to transform a declaration into objects in a scene treemanager, wherein thereafter such objects can be transformed into asecond declaration;

FIG. 2 is a block diagram of an embodiment of the present invention, inwhich an application system in a multiuser environment contains asimulation engine that, in turn, contains a scene tree object managerthat is hosting, in this example, a visual editor;

FIG. 3 is a block diagram of an embodiment of the present invention, inwhich an application system in a multiuser environment contains asimulation engine that, in turn, contains a scene tree object managerthat is hosting, in this example, a viewer;

FIG. 4 is flow diagram showing logical processes, responsive to a changein a scene tree, made by a first user in a multiuser environment, usinga visual editor, for propagating the change to other users in thenetwork in accordance with an embodiment of the present invention;

FIG. 5 is flow diagram showing logical processes by which a change, madeby a first user in a multiuser environment, is propagated over thenetwork and applied to a local scene tree experienced by a second userin accordance with an embodiment of the present invention;

FIG. 6 is a flow diagram showing logical processes by which any changemade in any declaration by any user on the network to any sceneaccessible over the network is synchronized with other users on thenetwork and with a declaration repository residing on a server system inaccordance with an embodiment of the present invention;

FIG. 7 is a block diagram of an embodiment of the present invention,showing how the server participates as a listener in a multiuserenvironment in a manner whereby the declaration repository associatedwith the server is updated with each change made in any declaration byany user on the network to any scene tree accessible over the network;

FIG. 8 is a block diagram showing logical processes carried out tocompile and execute a computer program in a typical prior art computerenvironment;

FIG. 9 is a block diagram showing logical processes carried out tocreate and execute a computer program in a computer environment inaccordance with an embodiment of the present invention;

FIG. 10 is a block diagram showing logical processes carried out in anembodiment of the present invention to provide a language extension,using instructions in a format going beyond a declaration, by invoking adynamic scripting language through a fire-walled environment via aprocedural call;

FIG. 11 is block diagram showing software components of the virtualizedsoftware environment, used in an embodiment of the present invention,that resides in the application system; and

FIG. 12 is block diagram of an embodiment of the present invention, inwhich an application system in a multiuser environment contains asimulation engine that, in turn, contains a scene tree object managerthat is hosting, in this example, declarations establishingfunctionality of a server system.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Definitions. As used in this description and the accompanying claims,the following terms shall have the meanings indicated, unless thecontext otherwise requires:

A “set” has at least one member.

A “computer process” is the performance of a described function in acomputer using computer hardware (such as a processor,field-programmable gate array or other electronic combinatorial logic,or similar device), which may be operating under control of software orfirmware or a combination of any of these or operating outside controlof any of the foregoing. All or part of the described function may beperformed by active or passive electronic components, such astransistors or resistors. In using the term “computer process” we do notnecessarily require a schedulable entity, or operation of a computerprogram or a part thereof, although, in some embodiments, a computerprocess may be implemented by such a schedulable entity, or operation ofa computer program or a part thereof. Furthermore, unless the contextotherwise requires, a “process” may be implemented using more than oneprocessor or more than one (single- or multi-processor) computer.

An “operating system” is an environment of a host computer system, whichmay be a conventional operating system, but alternatively may be anycomputing environment that can cause execution of computer instructions,receive input and provide output, including, for example, a web browser,a Java runtime environment, a Microsoft common language runtimeenvironment, etc.

A “cloud infrastructure” is an environment supporting input and outputover a network and by which there is caused execution of computerinstructions over the network.

A “virtualized” software environment is an environment by providing alayer of abstraction on top of a specific operating system or a specificcloud infrastructure, so that interaction with the virtualized softwareenvironment is independent of the specific operating system or thespecific cloud infrastructure.

An “application system” is a system selected from the group consistingof (i) a virtualized software system that is configurable to exist ontop of any desired operating system and (ii) a virtualized cloudinfrastructure that is configured to exist on top of any desired cloudinfrastructure and (iii) combinations thereof. A “simulation engine” isa software functionality, in the portion of the application system thatruns on top the

A “frame clock” is a timing mechanism defining successive frames withrespect to which operations of the simulation engine are synchronized.

A first computer system is in communication “in real time” over anetwork with a second computer system, when information is exchangedbidirectionally between the computer systems according to schedulingdetermined by the first and second computer systems and withoutdependency on polling.

A “declared scene tree” is a hierarchical map of objects and theirrelationships, properties, and behaviors, defined by a declaration, andthat are capable of being instantiated.

hierarchical characterization of objects and their relationships,properties, and behaviors in an instantiation of a declaration. In anygiven “instantiated scene tree” (also sometimes called a “scene graph”)is an abstract computing device, however, the instantiated scene tree isrendered, if at all, only in accordance with the virtualized softwareenvironment of the given computer device. Accordingly, when the givencomputing device is a replication computer, there is no rendering of theinstantiated scene tree at all.

A “visible scene tree” is a representation of objects, and theirrelationships, properties, and behaviors, in a corresponding scene tree,that are simultaneously visible in a display.

An “interactive scene tree” is a representation of objects, and theirrelationships, properties, and behaviors, in a corresponding scene tree,that are simultaneously available for user interaction in a display.

A “temporal assignment” is the specifying, effective at an initial time,of (i) a set of target values for a corresponding set of variables, (ii)a target time at which the target set of values is to be achieved, and(iii) and a time-varying function by which the target set of values isto be achieved. The time-varying function is operative over successiveframe counts to transform the initial set of values at the initial timeto the target set of values at the target time, using the ratio of the(a) amount of time remaining to reach the target time to (b) to thedifference between the target time and the initial time.

An “temporal procedure” is the specifying, effective immediately, of (i)a procedure and a set of parameters for the procedure, (ii) a targettime at which the procedure will be invoked, and (iii) optionally, anumber of times over which the procedure will be repeated.

A computer process defined by a temporal assignment or by a temporalprocedure is subject to a “software contract” when configured by theassignment to carry out the process under conditions selected from thegroup consisting of best efforts, guaranteed, and combinations thereof.

A “best efforts” software contract associated with a computer process iswhen the process is carried out on a frame-by-frame basis, wherein, iftasks defined for a given frame cannot be carried out, then processingmoves to tasks associated with the next frame even if the defined taskshave not been carried out.

A “guaranteed” software contract associated with a computer process iswhen the process is carried out on an absolute basis wherein tasksassociated with a target frame must be carried out before proceeding totasks associated with a succeeding frame.

“Linear logic” is a set of procedures that can be executed strictly fromleft to right, or top to bottom, without looping or branching, and thatcan be defined with zero, one, or more parameters.

An “declaration respository” of a repository server system in amultiuser network is a shared storage facility in which is stored a logof each change made in any declaration by any user on the network to anyscene tree accessible over the network and which communicates with anoff-line synchronizer of each client to provide, to such client, anychanges on the stored log that had not been synchronized when suchclient had been offline.

FIG. 1 is a flow diagram showing logical processes carried out by asimulation engine in accordance with an embodiment of the presentinvention to transform a declaration into objects in a scene tree objectmanager, wherein thereafter such objects can be transformed into asecond declaration. In this respect, the processes are “mirroring”processes, by causing the declaration to be mirrored in objects in thescene tree object manager and those objects in turn to be mirrored in asecond declaration. These processes begin with an input 101, which is adeclaration, namely a text-based representation of a set of objects.Embodiments of the present invention require a declaration to conform toa specific grammar, described below, that facilitates processing of thedeclaration. The declaration input 101 is provided to a declarationprocessor 103 that uses the declaration to create an abstract datastructure 105 that we term a “declared scene tree of objects.” Thedeclared scene tree of objects 105 includes abstract representations ofobjects such as classes and is used by the scene tree object manager 107to create an instantiated scene tree 109 of fully specified objects.

It can be seen that from the previous paragraph that a declaration candefine an instantiated scene tree of objects. Additionally, when thereis an instantiated scene tree of objects, in embodiments of the presentinvention, it is also possible to create a second declaration.Therefore, in accordance with processes carried out in embodiments ofthe present invention, the persistence processor 111 transforms some orall objects in the instantiated scene tree of objects 109 into a seconddeclaration 113. Accordingly, it can be seen that in embodiments of thepresent invention that a declaration is mirrored in an instantiatedscene tree of objects, and that the declaration and instantiated scenetree have a direct correspondence with one another.

Although a declaration may give rise to a complete instantiated scenetree of objects as just described, it is an aspect of embodiments of theinvention to use a partial declaration to characterize partial aspectsof an instantiated scene tree. In this way, one or more partialdeclarations can modify an instantiated scene tree in whole or in part,and vice versa: a partial aspect of an instantiated scene tree can beused to establish a second declaration corresponding to the partialaspect.

FIG. 2 is a block diagram of an embodiment of the present invention, inwhich an application system 201 in a multiuser environment contains asimulation engine 203 that, in turn, contains a scene tree objectmanager 107 that is hosting, in this example, a visual editor 205.

The Application System 201 interfaces with the Network Users 209 andpresents the application to the Local User 207. The Application System201 contains a Simulation Engine 203, which is responsible for theapplication behavior, a Frame Processor 221 to keep the simulation insynchronization with the application behavior, visual rendering and theother users and a Virtualized software environment 223, which abstractsa selected operating system or operating environments for use with theApplication System 201.

The Simulation engine 203 is responsible for the majority of work indelivering the application behavior. The Simulation Engine 203 containsa Declaration processor 103 and a persistence processor 111 responsiblefor converting objects in the scene tree to and from declarations. TheSimulation Engine 203 also contains a Frame Queue 219, to store softwarecontracts required by the simulation, and a Scene Tree Object Manager107, which manages all of the objects in the application.

The Scene Tree Object Manager 107 also contains an Instantiated SceneTree of Objects 109. These Objects exist in the running application. Theapplication is manifested to the local user 207 through the VisualEditor 205. The Visual Editor 205 in turn contains an InstantiatedEmbodiment 211 of a project. The Visual Editor 205 also contains anEditor Listener 213, to listen for network traffic and to makecorresponding changes to the instantiated scene tree of objects 109, andan Offline Sync 215 feature, to download changes made by other NetworkUsers 209 when the local user 207 has been offline. The Visual Editor205 also contains an Editor Broadcaster 217 to share, with the otherNetwork Users 209, any changes made by the Local User 207.

FIG. 3 is a block diagram, similar to that of FIG. 2, of an embodimentof the present invention, in which an application system in a multiuserenvironment contains a simulation engine that, in turn, contains a scenetree object manager that is hosting, in this example, a viewer, which isachieved by using the Visual Editor 305 in a read-only mode.

The Visual Editor 305 in FIG. 3 is similar to the Visual Editor 205 ofFIG. 2, but differs from the Visual Editor 205 in FIG. 2 in not usingthe Persistence Processor 111 and Editor Broadcaster 217, which areshown in dashed outline in FIG. 3. Because the Visual Editor 305,serving only as a viewer, is incapable of making edits to theinstantiated scene tree of objects 109, there are no edits of the localuser 207 to broadcast. Only instantiating a cached project or incomingupdates from the Network Users 209 are required for operation of theVisual Editor 305 as a viewer. All of the other elements required forthe correct simulation of the project are the same as in FIG. 2.

FIG. 4 is flow diagram showing logical processes, responsive to a changein a scene tree, made by a first user in a multiuser environment, usinga visual editor, such as Visual Editor 205 of FIG. 2, for propagatingthe change to other users in the network in accordance with anembodiment of the present invention. In FIG. 4, in process 401, user1uses the Visual Editor 205 to make a change to the instantiated scenetree of objects 109. In process 403, the Visual Editor 205 automaticallyrequests a declaration from the persistence processor 111 for mirroringthe change made by the first user. In process 405, the persistenceprocessor 111 gets the change from the Scene Tree Object Manager 107 andgenerates the declaration. In process 407, the Visual Editor 205 nowuses the Editor Broadcaster 217 to send the declaration to other NetworkUsers 209 as a broadcast.

FIG. 5 is flow diagram showing logical processes by which a change, madeby a first user in a multiuser environment, using a visual editor, suchas Visual Editor 205 of FIG. 2, is propagated over the network andapplied to a local scene tree experienced by a second user in accordancewith an embodiment of the present invention. In process 501 of FIG. 5,the Local Visual Editor 205 of FIG. 2 receives a change declaration froma Network User 209 via the Editor Listener 213 over the network. Inprocess 503, the Visual Editor 205 automatically requests thedeclaration processor 103 to mirror this change. In process 505, thedeclaration processor 103, via the Scene Tree Manager 107, then appliesthis change to the instantiated scene tree 109.

FIG. 6 is a flow diagram showing logical processes by which any changemade in any declaration by any user on the network to any sceneaccessible over the network is synchronized with other users on thenetwork and with a declaration repository residing on a server system inaccordance with an embodiment of the present invention. In process 601of FIG. 6, User 1 makes a change to the current project, for example,using Visual Editor 205 of FIG. 2. In process 603, this change is sentby the Editor Broadcaster 217 of FIG. 2 to the network users 209. Thechange sent by the Editor Broadcaster 217 is received in two differentcontexts. The first context is that of the visual editors of otherusers; thus, in process 605, an Editor Listener of User 2, similar toEditor Listener 213 of FIG. 2, receives the change, and, in process 607,communicates with the scene tree manager of User 2 to update the currentproject to reflect the change. The second context is that of the server.(We discuss the structure of the server in connection with FIGS. 7 and12 below.) In process 609 of FIG. 6, the Server Listener picks up thechange broadcast by User1 in process 601. In process 611, the ServerListener then records the change by storing it (in the declarationrepository 701 of FIG. 7) as part of a master project log.

FIG. 7 is a block diagram of an embodiment of the present invention,showing how the server participates as a listener in a multiuserenvironment in a manner whereby a declaration repository associated withthe server is updated with each change made in any declaration by anyuser on the network to any scene tree accessible over the network. InFIG. 7. the declaration repository 701 is a master project log residingon a server which is also running a Server Listener 705. The ServerListener 705 contains an Always Sync process 703, which works (i) toreceive any incoming change declarations from Network Users 209 of FIG.2, (ii) to resolve any conflicts, and (iii) to commit these changedeclarations to the declaration repository 701. The declarationrepository 701, in turn, can be used to load the project, to auditchanges, and to supply an update to users who have been offline.

FIG. 8 is a block diagram showing logical processes carried out tocompile and execute a computer program in a typical prior art computerenvironment. In FIG. 8, these processes are exemplary, as there are manytypes of traditional compiler and virtual machine designs. The typicalprior art shares a common thread, however, which involves two steps: (1)compiling and (2) executing. Broadly, compiling is the process ofconverting textual code into a lower level tree of instructions,optimization and linking to external functions. Then the finalizedinstructions are exported to a platform specific binary or executeddirectly inside a virtual machine.

In FIG. 8, in process 801, a textual representation of a computerprogram is loaded into a computer. In process 803, the program is thenlexically analyzed and in processes 809 and 805 respectively, tokens andsymbol table are extracted. In process 807, the parser uses the SymbolTable to build a Syntax Tree 813, which is a hierarchical representationof the elements of the code in the textual program provided as an input.The Syntax Tree 813 is then put through an Intermediate code generationprocess 811 to generate the Intermediate code instructions 815describing the program. The intermediate code instructions are in aformat suitable for being operated on for platform translation, avirtual machine or an optimizing compiler. In process 817, theintermediate code is used for generating machine code and in process 819the machine code is subject to instruction level optimization to providemachine specific instruction code 821. This code is linked to otherlibraries in process 823 to provide a platform specific binary 825 builtas the final executable. Then, when the platform specific binary isexecuted in process 827, the operating system API's are used torequisition GUI and other resources in process 829 to deliver a platformspecific operating system and machine instruction version of theprogram.

FIG. 9 is a block diagram showing logical processes carried out tocreate and execute a computer program in a computer environment inaccordance with an embodiment of the present invention. Embodiments ofthe present invention utilize simulation of a computer program in amanner that is very different from the normal processes described inconnection with FIG. 8. Whereas in the prior art, as discussed inconnection with FIG. 8, execution is essential for a normal program tooperate on a computer, embodiments of the present invention provide foroperation on a computer via a novel simulation mode. In theseembodiments, foundational software is first compiled and executed in thenormal manner for a specific machine instruction set and operatingsystem and associated API's. This foundational software establishes theapplication system 201 described in connection with FIG. 2.

After the application system 201 is loaded into a computer, theprocesses of FIG. 9 can be carried out. In process 903, the user'sprogram, in the form of a declaration or source code in machine readableform corresponding to a declaration, is received by the applicationsystem 201. In process 904, the loaded declaration is lexically analyzedand parsed by the declaration processor (item 103 in FIG. 1) and turnedinto a Declared Scene Tree of Objects (item 105 of FIG. 1).

-   -   The declaration syntax and language constructs are intentionally        constrained to make declaration processing, in process 904,        rapid and efficient. Specifically, declarations are required to        adhere to the following rules of grammar:    -   (1) a loop is not permitted;    -   (2) a branching condition is not permitted;    -   (3) a procedure with a set of parameters is permitted (including        a procedure call to a procedural function in a scripting        language suitably registered with a procedural contract); and    -   (4) an object in a declared scene tree is permitted to have a        set of states, wherein:

each state must be separately selected from a group consisting of (a) astate that is a set of nested states and (b) a state that isnonoverlapping with any other state; and each state can encompass a setof properties, a set of events, and a set of sub-objects; and

a given state can only be active or inactive; and

active states in a declaration are applied, by the scene tree manager,in the order in which they are declared.

The effect of these rules is that when in process 905 the scene treeobject manager 107 of FIG. 2 instantiates the objects defined by thedeclaration to create a final Instantiated Scene Tree of Objects 109, itforms a simple directed acyclic graph. This property allows the VisualEditor 205, in process 907 to edit the live Scene Tree, and, in process911, for many real time editors to participate. This approach of keepinga live model of the running application as a graph allows for multi userediting and high level optimizations to be applied, because, in process901, the program is simulated. Finally, in process 909, the persistenceprocessor can be used to save the current live state of the InstantiatedScene Tree of Objects 109. The optimization process 901 is able toperform a huge number of optimizations such as resource caching,pre-caching, level of detail, lazy evaluation, computation caching andocclusion testing. Various embodiments herein provide a visual scenetree and a goal-directed model in the simulation, so the running programexplicitly knows the current and near future goals and can avoid typicalprocessing overhead associated with the prior art—while still ensuringany contracts are completed on time using the Frame Processor 221.

FIG. 10 is a block diagram showing logical processes carried out in anembodiment of the present invention to provide a language extension,using instructions in a format going beyond a declaration, by invoking adynamic scripting language through a fire-walled environment via aprocedural call. In the embodiment of FIG. 10, there is employed aloosely coupled procedural contract 1005 to allow a traditional compiledor just in time compiled language (Javascript in this example), to beused as an extension of the declaration grammar described in connectionwith FIG. 9. The procedural contract 1005 accesses the JavaScript orother compiled or scripted function, in box 1003, that has the featuresdiscussed above in connection with FIG. 8, which are shown as componentsof box 1003. This embodiment allows developers to write new JavaScriptfunctions and expose them as a visual linear action in a mannerconsistent with the declaration grammar. The procedural contract is anagreement on the name of the function, its parameters and their types,and any callback events which may be required to handle the output orerrors that may occur. These events are bubbled back into thedeclaration processing as normal actions. The benefit for thisembodiment is that when Javascript is packaged as a procedural contract,the resultant action still meets the restricted syntactical andstructural definitions of the FIG. 9 embodiment while allowingdevelopers to write arbitrarily complex external code.

The second half of the challenge posed in this embodiment is to enablethese JavaScript programs to do anything of consequence via theapplication system 201 of FIG. 2. This functionality is made possiblethrough the Functional Gateway 1007 of FIG. 10. The Functional Gateway1007 allows the JavaScript programmer to safely and asynchronouslyaccess the operating system and hardware via the Virtualized SoftwareEnvironment 223. The Functional Gateway 1007 lets the Javascriptprogrammer play sounds, access the network or other API's via a genericcross-platform API. The Functional Gateway 1007 also lets the Javascriptprogrammer safely and asynchronously access the running applicationobjects and make changes using the Scene Tree Object Manager 107.

FIG. 11 is block diagram showing software components of the virtualizedsoftware environment 223, used in an embodiment of the present inventionthat resides in the application system of FIG. 2. The VirtualizedSoftware Environment 223 sits on top of the specific combination ofdevice hardware, operating system, input devices and output devices1103. The goal is to virtualize and generalize the computing environmentwhich may be client or server devices. This means for clientenvironments the traditional operating system GUI elements are not usedand a 3D rendered surface is provided instead. As a result theenvironment created by this embodiment has the ability to make quitedifferent systems look more similar. There are many different types ofobjects in the Virtualized Software Environment and the presentembodiments are specifically designed to allow these objects (indicatedgenerically as item 1113) to be quickly extended as new capabilities arereleased by device and OS manufacturers. Some examples of virtualizedobject types include loading and displaying Images 1105, Rendering andallowing input for Text 1107, HTTP network protocols 1109, Mouse/Touch1111.

FIG. 12 is block diagram of an embodiment of the present invention, inwhich an application system in a multiuser environment contains asimulation engine that, in turn, contains a scene tree object managerthat is hosting, in this example, declarations establishingfunctionality of a server system. The block diagram of the server systemis much simpler than the visual editor in FIG. 2 as it excludes thevisual elements. In this case the Virtualized Software Environment 223and FIG. 11 would also be quite different as the objects would benon-visual and server-specific. The Application System 201 for theserver further contains a Simulation Engine 203 with a Server Listener1201 receiving new project updates from Network Users 209 to theInstantiated Scene Tree of Objects 109 processed by the DeclarationProcessor 103 and applied by the Scene Tree Object Manager 107.

The present invention may be embodied in many different forms,including, but in no way limited to, computer program logic for use witha processor (e.g., a microprocessor, microcontroller, digital signalprocessor, or general purpose computer), programmable logic for use witha programmable logic device (e.g., a Field Programmable Gate Array(FPGA) or other PLD), discrete components, integrated circuitry (e.g.,an Application Specific Integrated Circuit (ASIC)), or any other meansincluding any combination thereof.

Computer program logic implementing all or part of the functionalitypreviously described herein may be embodied in various forms, including,but in no way limited to, a source code form, a computer executableform, and various intermediate forms (e.g., forms generated by anassembler, compiler, networker, or locator.) Source code may include aseries of computer program instructions implemented in any of variousprogramming languages (e.g., an object code, an assembly language, or ahigh-level language such as Fortran, C, C++, JAVA, or HTML) for use withvarious operating systems or operating environments. The source code maydefine and use various data structures and communication messages. Thesource code may be in a computer executable form (e.g., via aninterpreter), or the source code may be converted (e.g., via atranslator, assembler, or compiler) into a computer executable form.

The computer program may be fixed in any form (e.g., source code form,computer executable form, or an intermediate form) either permanently ortransitorily in a tangible storage medium, such as a semiconductormemory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-ProgrammableRAM), a magnetic memory device (e.g., a diskette or fixed disk), anoptical memory device (e.g., a CD-ROM), a PC card (e.g., PCMCIA card),or other memory device. The computer program may be fixed in any form ina signal that is transmittable to a computer using any of variouscommunication technologies, including, but in no way limited to, analogtechnologies, digital technologies, optical technologies, wirelesstechnologies, networking technologies, and internetworking technologies.The computer program may be distributed in any form as a removablestorage medium with accompanying printed or electronic documentation(e.g., shrink wrapped software or a magnetic tape), preloaded with acomputer system (e.g., on system ROM or fixed disk), or distributed froma server or electronic bulletin board over the communication system(e.g., the Internet or World Wide Web).

Hardware logic (including programmable logic for use with a programmablelogic device) implementing all or part of the functionality previouslydescribed herein may be designed using traditional manual methods, ormay be designed, captured, simulated, or documented electronically usingvarious tools, such as Computer Aided Design (CAD), a hardwaredescription language (e.g., VHDL or AHDL), or a PLD programming language(e.g., PALASM, ABEL, or CUPL).

While the invention has been particularly shown and described withreference to specific embodiments, it will be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the invention asdefined by the appended clauses. While some of these embodiments havebeen described in the claims by process steps, an apparatus comprising acomputer with associated display capable of executing the process stepsin the claims below is also included in the present invention. Likewise,a computer program product including computer executable instructionsfor executing the process steps in the claims below and stored on acomputer readable medium is included within the present invention.

The embodiments of the invention described above are intended to bemerely exemplary; numerous variations and modifications will be apparentto those skilled in the art. All such variations and modifications areintended to be within the scope of the present invention as defined inany appended claims.

What is claimed is:
 1. A nontransitory storage medium encoded withinstructions, for use with a plurality of computer systems, each one ofwhich is in communication in real time over a network with othercomputer systems of the plurality of computer systems, wherein theinstructions, when executed on each of the computer systems, establish,on each computer system, an application system, wherein the applicationsystem includes a simulation engine and a virtualized softwareenvironment, the simulation engine running on top of the virtualizedsoftware environment and enabling creation and editing of a project by alocal user and network users, wherein the simulation engine comprises: adeclaration processor that linearly processes a first declaration thatconstitutes a text embodiment of the project, reciting any set of itemsselected from the group consisting of object, relationship, property,behavior, and combinations thereof, into a declared scene tree ofobjects; a scene tree object manager that (a) manages objects, theirrelationships, properties, and behaviors in the declared scene tree ofobjects, (b) causes transformation of the declared scene tree of objectsso as to be mirrored in a project subnode of an instantiated scene treeof objects, the project subnode constituting an instantiated embodimentof the project, and (c) causes updating of the instantiated scene treewhen it is changed; a persistence processor in communication with thescene tree object manager, that causes mirroring, of the instantiatedscene tree of objects, in a second declaration; a visual editor, thevisual editor being a subnode of the instantiated scene tree, that makesuser-directed changes to the project subnode of the instantiated scenetree, wherein the project subnode is a subnode of the visual editor; aneditor broadcaster, the editor broadcaster being a subnode of theinstantiated scene tree, coupled to the network, that obtains changesfrom the visual editor and causes the persistence processor to transforma correspondingly changed part of the instantiated scene tree into achange declaration and broadcasts the change declaration embedded in achange message over the network to the other computer systems; and aneditor listener, the editor listener being a subnode of the instantiatedscene tree, coupled to the network, that receives and processes anychange message, from any other of the computer systems, by causing thedeclaration processor to transform a declaration embedded in receivedchange message into a corresponding change in the instantiated scenetree; a rendering processor, coupled to the virtualized softwareenvironment, to requisition hardware resources to cause physicalmanifestation of the instantiated scene tree of objects in a formselected from the group consisting of audio, visual, networkcommunication, storage in a transitory or a nontransitory basis,computational tasks, and combinations thereof, the physicalmanifestation being optimized by evaluation of hardware resources thatare both available and required to be allocated in order to achieve thephysical manifestation.
 2. A storage medium according to claim 1,wherein the editor has a mode wherein it operates solely as a viewer anddoes not communicate with the editor broadcaster or the persistenceprocessor.
 3. A storage medium according to claim 1, wherein the visualeditor maintains a globally unique identifier for each object in theinstantiated scene tree, so that each change, made over the network, canuse the identifier to reference the corresponding object and that suchchange can be correctly applied to the instantiated scene tree.
 4. Astorage medium according to claim 1, wherein the visual editorassociates, with each change made by a user over the network, anidentification of the user making the change and a time stamp.
 5. Astorage medium according to claim 1, wherein the visual editorcharacterizes each change, made over the network, to the instantiatedscene tree, as a member of the group consisting of insertion, deletion,update, lock, unlock, and combinations thereof.
 6. A storage mediumaccording to claim 1, wherein each of the computer systems is coupledover the network to a server system, and the editor broadcaster alsobroadcasts the change declaration embedded in the change message overthe network to the server system for storage thereof in a declarationrepository.
 7. A storage medium according to claim 1, wherein thesimulation engine further comprises: a frame clock defining consecutiveframes of the engine with respect to which operations are synchronized.8. A storage medium according to claim 1, wherein the simulation enginefurther comprises: a frame processor, coupled to the virtualizedsoftware environment and to the simulation engine, to require thesimulation engine to perform operations, specified with respect to asequence of frames using the frame clock, to assure that the processesare performed sequentially.
 9. A storage medium according to claim 8,wherein (a) the simulation engine further comprises a frame queue,filled by the simulation engine, in which are stored, in frame order,software contracts pertinent to processes to be carried out by thesimulation engine, and (b) the frame processor is coupled to the framequeue and, if a specific software contract in the frame queue reaches acurrent frame for performance, the frame processor enforces the terms ofthe specific software contract as to best efforts and guaranteed terms.10. A storage medium according to claim 9, wherein the frame processoris configured to process, concurrently with the current frame, a set offrames sequenced after the current frame, with respect to demandexpected to be placed on system resources by the specific softwarecontract, so as to enhance efficiency of execution of the contract wheneach frame in the set of frames shall have become the current frame. 11.A storage medium according to claim 10, wherein, in the processing of anevent, via the frame processor, from the virtualized softwareenvironment, the simulation engine executes logic creating a temporalassignment, which is added to the frame queue at a frame numbercomputed, based on a current frame number and a target time, in frameorder by object and property, so that if the computed frame number isless than a frame number of any second item in the frame queue, then thesecond item will be flushed from the queue.
 12. A storage mediumaccording to claim 11, wherein, at the time of creating the temporalassignment the frame processor stores an initial set of values and theinitial time along with a target set of values and a target time, and aninterpolation function, when a frame needs to be computed, the frameprocessor computes a set of current values using the current time, theinitial time and the target time to drive the interpolation functionbetween the initial values and target values.
 13. A storage mediumaccording to claim 12, wherein the frame processor operates on a bestefforts basis to cause updating of the interactive scene tree and visualscene tree so as to correspond with the calculated set of values untilthe final frame has been, at which time the frame processor causes thetemporal assignment to be contractually updated throughout the system.14. A storage medium according to claim 10, wherein, in the processingof an event, via the frame processor, from the virtualized softwareenvironment, the simulation engine executes logic creating a temporalprocedure, which is added to the frame queue at a frame number computed,based on a current frame number and a target time, in frame order byobject and property, so that if the computed frame number is less than aframe number of any second item in the frame queue, then the second itemwill be flushed from the queue.
 15. A storage medium according to claim1, wherein the first declaration conforms to a declaration grammarwherein: a loop is not permitted; a branching condition is notpermitted; a procedure with a set of parameters is permitted; and anobject in a declared scene tree is permitted to have a set of states,wherein: each state must be separately selected from a group consistingof (a) a state that is a set of nested states and (b) a state that isnonoverlapping with any other state; and each state can encompass a setof properties, a set of events, and a set of sub-objects; and a givenstate can only be active or inactive; and active states in a declarationare applied, by the scene tree manager, in the order in which they aredeclared.
 16. A storage medium according to claim 15, wherein thepermitted procedure with the set of parameters includes a procedure callto a procedural function in a scripting language.
 17. A nontransitorystorage medium according to claim 1, wherein the scene tree isconfigured to cause the second declaration is to replace the firstdeclaration.
 18. A nontransitory storage medium according to claim 1,wherein the simulation engine includes a library of classes thatimplement an abstraction of a set of operating systems.
 19. Anontransitory storage medium according to claim 1, wherein the editorlistener of any given client includes an off-line synchronizer coupledto a repository server system, the repository server system having adeclaration repository that stores a log of each change made in anydeclaration by any user on the network to any scene tree accessible overthe network, so that, when the given client comes online after havingbeen offline, the off-line synchronizer communicates with thedeclaration repository to synchronize changes that were not synchronizedwhen the given client had been offline.
 20. A nontransitory storagemedium encoded with replicating instructions, for use on a computerconfigured as a replicating computer in the plurality of computersystems of claim 1, the replicating instructions, when executed on thereplicating computer, establish, on the replicating computer, areplicating application system, wherein the replicating applicationsystem includes a replicating simulation engine and a replicatingvirtualized software environment, the replicating simulation enginerunning on top of the replicating virtualized software environment,wherein the replicating simulation engine comprises: a replicatordeclaration processor that linearly processes a first declaration thatconstitutes a text embodiment of the project, reciting any set of itemsselected from the group consisting of object, relationship, property,behavior, and combinations thereof, into a declared scene tree ofobjects; a replicator scene tree object manager that (a) managesobjects, their relationships, properties, and behaviors in a declaredscene tree of objects, (b) causes transformation of the declared scenetree of objects so as to be mirrored in a project subnode of aninstantiated scene tree of objects, the project subnode constituting aninstantiated embodiment of the project, and (c) causes updating of theinstantiated scene tree when it is changed; a replicator editor, thatmakes user-directed changes to the project subnode of the instantiatedscene tree, wherein the project subnode is a subnode of the visualeditor; and a replicator listener, coupled to the network, that receivesand processes any change message, from any other of the computersystems, by causing the declaration processor to transform a declarationembedded in received change message into a corresponding change in theinstantiated scene tree.